STUDY 1: THE EFFICACY OF INTERACTIVE PRESENTATION

A pilot study was conducted to verify the expectation that the interactive visual presentation of stimuli in the VAT induces robust incidental learning.

Methods

Subjects

The subjects were residents of homes for the elderly who volunteered for the study through mediation of the staff. The influence of cognitive deterioration was reduced by excluding subjects who scored below the 10th centile on the cognitieve screening test,¹⁰ a Dutch adaptation of the short portable mental status questionnaire.¹¹ The data for the two conditions (A and B; see below) were collected independently, which led to considerable differences in group composition. However, the groups were comparable in age (table 1). Educational attainment, cognitieve screening test score, and the percentage of women were all higher in group B than in group A.

Procedure

Group A received the VAT as described above with interacting stimuli and no explicit instructions to memorise the items. In group B, the associations were shown as pictures of two separate objects. The subjects of group B were instructed to remember that the newly shown (target) object went together with the previously shown (cue) object. Each pair was presented once, but as much time as desired was allowed to rehearse the combinations. In other respects (presentation of cue cards, testing of recall) the procedure was the same for both groups.

Results

The median number of items recalled was five out of six in group A (interactive presentation), but only two out of six in group B (side by side). The difference is highly significant (p<0.0001, Mann-Whitney U test).

Discussion

Aside from the manner of presentation, the conditions for recall were more favourable for group B (side by side presentation) than for group A (interactive presentation), because group B was not only given learning instructions and the opportunity to rehearse the items, but was also more educated and obtained higher mental status scores. It is therefore all the more notable that recall was much better in group A. We conclude that the interactive presentation of visual paired associates induces a degree of incidental learning that surpasses intentional learning of the same stimuli when presented in the traditional (side by side) fashion.

STUDY 2: THE VAT IN A COMMUNITY BASED SAMPLE

The following describes a study on the detection of DAT in a community based sample, comparing the VAT with the mini mental state examination (MMSE)¹² and another six item visual test of incidental learning.

Methods

Subjects

The subjects were participants of the Amsterdam study of the elderly (AMSTEL), a follow up study on cognitive decline in community dwelling elderly persons aged 65–84 years. In the screening phase, a source population of 5666 people was recruited from 30 general practices spread throughout the city. Selection was random except for stratification into four five year age groups. The response rate was 71.5% (n=4051). All subjects took part on the basis of informed consent. A cohort of 511 was chosen, maintaining age stratification but the sample was selectively weighted to include mostly people with lower MMSE scores. All subjects with MMSE scores below 22 points were included, but only one third of those in the 22–26 range and 6% of those with a higher score. More details on the design and selection procedure of AMSTEL can be found elsewhere.¹³

Diagnostic procedures and classification

The cohort members received four yearly diagnostic examinations over a three year period. The follow ups will henceforth be designated as years 1–3. The examinations included the Cambridge examination for mental disorders of the elderly (CAMDEX),¹⁴ the geriatric mental state exam (GMS),¹⁵ and a short medical evaluation. On the basis of these data, the subjects were diagnosed by joint decision of the examining nurse and doctor and the study neurologist (CJ), using DSM-III-R criteria for dementia and NINCDS-ADRDA criteria for probable DAT. For the present purpose, we selected 249 subjects, divided into three groups:

1. DAT (n=24): incident cases, diagnosed with DAT in one of the follow ups (12 in year 1 and six in years 2 and 3).

2. Declining (n=21): subjects who were not diagnosed with dementia within the study period, but lost more than 11 points on the CAMCOG (the cognitive section of the CAMDEX) between baseline and year 3. This decline exceeds the 99% confidence critical change score.¹⁶

3. Non-demented (n=204): subjects who were not diagnosed with dementia on year 3, nor showed cognitive decline as defined above. People with a positive psychiatric or neurological history other than dementia were not excluded.

This selection excluded patients with DAT who were diagnosed at baseline (n=36), cases of non-Alzheimer dementia (n=10), subjects who dropped out of the study without a diagnosis of dementia (n=207), and subjects with missing values on the study variables (n=9). Depression ratings were derived from the GMS with the AGECAT computer program¹⁷; the symptom levels were converted into an ordinal scale, with psychotic and severe neurotic depression coded as 4. Educational attainment was rated on an ordinal scale ranging from 1 (incomplete primary) to 7 (university); 2 points on this scale indicate a completed primary education, and 4, 5, and 6 points indicate a completed lower, middle, and higher secondary education respectively.

Table 2 shows group characteristics. The non-demented group differed from the other groups in mean age and baseline MMSE score; other differences were not significant.

Neuropsychological tests

Variables used to assess the discrimination between groups were the VAT, the MMSE, CAMCOG picture recall, CAMCOG picture recognition, and age. The VAT scores were independent from the diagnostic procedures, whereas the results from the other measures were incorporated into the diagnostic process. Like the VAT, the CAMCOG picture test measures incidental learning, but reproduction is not cued and is tested after a delay. The subject is asked to name six photographs of objects. Free recall is tested unexpectedly after the administration of seven other CAMCOG items. Next, recognition is tested by choice from photographs depicting three specimens of the object in question.

Statistical analysis

Firstly, we compared the test variables with regard to distribution, confounding influences, and reliability. Secondly, the discriminative capacities of the tests were evaluated with receiver operating characteristics (ROC) curves. Sensitivity was examined in three subject categories: DAT at the time of diagnosis, the same patients a year before their diagnosis, and declining subjects at baseline. In all three analyses, specificity was based on the baseline scores of non-demented subjects. Age was used to set minimal requirements for discrimination, reasoning that (the cut point of) a test variable is useful only when it has better sensitivity/specificity than a rule assigning all subjects above a certain age to the abnormal group. The ROC curve for age consists of all points classifying subjects above a given age (at the time of examination) as abnormal.

Results

Psychometric characteristics

Table 3 lists the frequencies of scores on the memory tasks in non-demented subjects at baseline. Because of the oversampling of subjects with low MMSE scores, the frequencies of low scores are unrepresentative of the whole population. Picture recall is more or less normally distributed and has a floor effect, limiting specificity to a maximum of 96.6%. The distributions of picture recognition and the VAT are highly skewed, with a somewhat more pronounced ceiling effect in the latter. Although dementia was excluded, this group may have contained some cases of amnesia from other causes. The one subject who scored 1 point on the VAT was known to have suffered anoxia during a heart attack, and one subject who scored 2 points had suffered a posterior cerebral artery stroke.

Within the non-demented group, test scores and subject characteristics were unrelated, except for a significant rank correlation between picture recognition and age (r=−0.25, p<0.01), MMSE and education (r=0.35, p<0.001), and MMSE and depression rating (r=−0.18, p<0.05).

The reliability of the test variables was measured by the correlation between the first and second administrations (interval one year) in the total sample and the three subgroups (table 4). Despite its shortness, the VAT is comparable to the MMSE in overall reliability. In the non-demented group, the low coefficient of the VAT is attributable to restricted variance (the SD was 1.1 points). The low reliabilities in the declining group may be due to subjects deteriorating at varying rates. The internal consistency of the VAT (Cronbach's α) in the total sample was 0.88.

Predictive validity

Figure 2 shows ROC curves for the distinction between test performance of patients with DAT at the time of their diagnosis and non-demented controls at baseline. The VAT discriminated slightly more effectively than the MMSE, reaching optimal results with cut point 2/3 (87.5% sensitivity, 97.5% specificity). Picture recognition (not shown) did not discriminate better than age at any level.

In fig 3, sensitivity is based on the scores of the patients with DAT, which were obtained a year before diagnosis. Here, the MMSE is less effective than either the VAT or picture recall, and it hardly discriminates any better than age. Discrimination is optimal with VAT cut points 2/3 (66.7% sensitivity, 97.5% specificity) and 3/4 (83.3% sensitivity, 90.7% specificity). ROC curves for the discrimination of the declining subjects are not shown because it cannot be determined how many of these actually developed DAT. However, it is noted that the difference at baseline between the declining group and the non-demented group was highly significant for the VAT (p<0.0001, Mann-Whitney U test) whereas the effect was weak for the MMSE (p=0.042) and not significant for picture recall and recognition.

Discussion

The results support our central hypothesis that the VAT detects the memory impairment that is characteristic for DAT. In the diagnosable stage, most patients with DAT score at a level that is rare in persons without dementia. More important, however, is that most DAT cases in this study could already be detected a year before diagnosis with equally high specificity (although with lower sensitivity). Whether prediction is efficient over a longer period remains to be shown. The low average of the declining group suggests that, in at least some cases, the VAT score starts deteriorating more than three years before the diagnosis.

This study was limited by the exclusion of cases of non-Alzheimer dementia. The question whether and to what extent the VAT is failed by such patients is addressed in study 3.

STUDY 3: THE VAT IN CLINICAL PRACTICE

In the second study we showed that the VAT is a sensitive and specific indicator of DAT and its prodromal stage. However, the incidence of dementia in the sample of study 2 was low compared with clinical settings, and the few patients who had non-Alzheimer types of dementia were excluded. Therefore, the aim of the third study was to show that a disorder of this kind of incidental learning is also rather specific for DAT with respect to other types of dementia.

Methods

Subjects

Subjects were patients in the memory clinics and the neurological and geriatric departments of a university hospital and a teaching hospital. All patients were referred for neuropsychological examination because the referring family doctor, geriatrician, or neurologist suspected a dementia syndrome. A total of 224 patients (57.1% female) were included between September 1996 and March 2000. Their mean age was 76.6 years (SD 7.3; range 51–91), and mean MMSE score was 23.0 (SD 3.8; range 11–30). Median educational level was 4 on a 7 point scale (see legend to table 1), corresponding to about 10 years of full time education; the interquartile range was 2–5 (about 6–11 years of education).

Diagnostic procedures and classification

All patients were examined at least once by a neurologist or a geriatrician, who carried out the additional laboratory, electroencephalographic, and neuroimaging investigations deemed necessary according to clinical consensus guidelines.¹⁸ After the history taking, physical and neurological examination, all patients were tested for haemoglobin, thyroid function, renal function, hepatic function, calcium, glucose, syphilis serology, and thiamin, folate, and vitamin B12 deficiency. Other investigations were carried out if indicated. All patients were evaluated by a neuropsychologist (see below). Most cases were discussed in the multidisciplinary staff meetings of the memory clinics.

For the purpose of this study, the clinical diagnoses were checked by strictly applying published consensus criteria on the information available in the patients' files. All clinical inclusion and exclusion criteria of the following five disorders were used:

1. Alzheimer's disease: DSM-IV criteria and NINCDS-ARDRA criteria for probable and possible DAT^19,20;

2. vascular dementia (VaD): DSM-IV criteria and NINDS-AIREN criteria for probable and possible VaD ^19,21;

3. frontotemporal lobar degeneration (FTD)²²;

4. dementia with Lewy bodies (DLB)²³;

5. subcortical dementia.^24,25

These criteria were recorded by a clinical neuropsychologist (BS) from the medical files within a time frame of three months before until one month after the neuropsychological examination. This was supplemented with information from the patient history and behavioural observation sections of the psychological report. Neuropsychological test results were not used for diagnostic classification (except the MMSE score, which was used as one of the NINCDS-ARDRA criteria). If a patient received diagnoses from more than one of these five disease categories, diagnoses of VaD, FTD, or DLB were given priority (in this order) over other diagnoses; a diagnosis of DAT was given priority over a diagnosis of subcortical dementia. Any inconsistencies between the clinical diagnosis and this diagnostic check were resolved by a geriatrician (LT) and a neurologist (GW), who reviewed the cases concerned (except their neuropsychological test results) to decide on the final diagnostic classification. A second file review was necessary in 21 cases; in 15 of these cases the original clinical diagnosis was maintained in favour of the result of the diagnostic check.

Neuropsychological tests

All patients were evaluated by an experienced neuropsychologist. A flexible approach was used; tests were applied as appropriate in view of the referral question and the characteristics of the patient. However, the following two memory tests were always used. The VAT was administered in the way described above. The prose recall subtest of the Rivermead behavioural memory test was also included in the evaluation.^26,27 In this test, a short, 21 item newspaper article is read to the patient, who is instructed to repeat as many items as possible immediately after presentation and after a median interval of 13 minutes (interquartile range 10–15 minutes). The MMSE score was obtained from the medical files. It was used in the diagnostic classification and as a measure of disease progression.

Statistical analysis

Firstly, the basic characteristics and MMSE scores of the diagnostic groups were compared by analysis of variance and non-parametric tests. Secondly, the memory test scores of the diagnostic groups were compared by analysis of variance, covarying for disease progression (MMSE). Analyses of variance were followed up with Bonferroni corrected post hoc analyses, accepting group differences with p<0.0125 as significant.

It was expected that the DAT group would score worse on the VAT than the other groups, and that this was not (or to a lesser extent) the case for immediate and delayed prose recall. Finally, a logistic regression analysis was performed with DAT versus non-DAT dementia as the dependent variable and with VAT score and the immediate and delayed prose recall scores as independent variables (method stepwise forward). The hypothesis was that the VAT would discriminate better than immediate or delayed prose recall.

Results

Of all patients included, 116 (51.8%) were diagnosed according to any of the sets of consensus criteria; the remaining patients either were not given a clinical diagnosis of a dementia syndrome or had insufficient data in their medical records to perform a valid diagnostic check. Thirty nine patients (34% of 116) received diagnoses from more than one of the disease categories. Table 5 shows the distribution of subjects over the diagnostic categories, their basic characteristics, and MMSE scores.

The groups were not significantly different with respect to basic variables, except sex distribution. There were more men in the FTD and subcortical subgroups than in the other groups. The MMSE scores of the diagnostic groups were not significantly different from each other (p=0.13), indicating that the groups were roughly comparable with respect to disease severity. Table 6 shows the results of the VAT and the prose recall test. As expected, there were significant differences in VAT score between the diagnostic groups. Post hoc comparisons showed that patients with DAT scored significantly worse than those with VaD, FTD, or subcortical dementia. The subgroup differences in immediate and delayed prose recall were also significant, but these differences were less pronounced. On immediate prose recall, patients with DAT scored significantly lower than those with subcortical dementia; on delayed prose recall they scored significantly lower than patients with VaD or subcortical dementia (p<0.05 Bonferroni corrected). In the logistic regression analysis, the VAT score was selected first (p=0.004) and the delayed prose recall score second (p=0.02); immediate prose recall was not included in the model (p=0.89). The analysis correctly classified 73% of DAT cases and 75% of non-DAT cases. Some 90% of patients with DAT scored 3 points or less (sensitivity), whereas 50% of those with non-DAT dementia scored 4 points or more (specificity). Sensitivity and specificity were 79% and 69% respectively for VAT scores of 2 or less, and 63% and 80% for VAT scores of 1 point.

Discussion

The sensitivity for DAT of VAT scores below 3 points found in this study (79%) was similar to that in study 2 (87.5%). The specificity of this cut point for the distinction between DAT and other types of dementia was 69%.

To prevent circularity in the study, we ignored the results of the neuropsychological evaluation when composing the diagnostic groups, but this may have led to classification errors. Other classification errors may have arisen from overlap between criteria sets, many patients satisfying more than one set of criteria. This overlap is a notorious problem.²⁸ We had to make somewhat arbitrary decisions to solve this problem. We gave preference to diagnoses of VaD, FTD, and DLB above DAT, because DAT is essentially a diagnosis by exclusion. Consequently, we reasoned that, if there are arguments in favour of VaD, FTD, or DLB, then they should have preference over DAT. Moreover, possible mixed pathology was ignored, notwithstanding the fact that this is common.^29,30 However, our strategy resulted in a relatively pure DAT group. The diagnostic check differed from the clinical diagnosis of DAT in only two cases (which had to be reclassified after revision). As our main goal was to investigate whether the VAT is able to specifically discriminate DAT from other types of dementia, some “pollution” of the other diagnostic groups was of minor importance. Moreover, the decision procedure ensured, on the one hand, that the results were not biased in favour of the DAT group and, on the other hand, maximised the number of patients in the non-DAT groups, which was regrettably small in some cases. Nevertheless, the “signal” in our findings remained clearly visible against the diagnostic “noise”. Our diagnostic groups were comparable in age, educational level, and dementia severity as expressed by the MMSE scores. Therefore differences in VAT scores cannot be ascribed to these potentially confounding influences. It may be claimed that the MMSE is a rather blunt instrument, and that equal MMSE scores do not completely rule out the possibility that the patients with DAT scored lower on the VAT because their dementia was more severe. However, the MMSE is not very sensitive to the more prominent symptoms of non-Alzheimer dementia, such as slowing and executive disorders.³⁰ Therefore, given equal MMSE scores, the disease is likely to be more advanced in non-Alzheimer patients than in patients with DAT.

The DLB group contained more patients with a low VAT score, and the mean VAT scores of patients with DLB were not significantly different from those of patients with DAT. Thus, we could not confirm the specificity of the VAT with respect to the distinction between DAT and DLB. This was an unexpected finding because a clear memory disorder is not an obligatory criterion for DLB. In fact, memory function in this disorder has been reported to be only mildly disturbed.^31,32 An explanation for our finding may be that the clinical distinction between DAT and DLB is particularly difficult.^33,34 In particular, the criterion of fluctuations in attention and alertness may be hard to establish in a reliable way.^35,36 This may have caused more misclassifications in the DLB group than in the other groups.

GENERAL DISCUSSION

The design of the VAT was based on the finding that the use of imagery mnemonics leads to a remarkable improvement in paired associate learning.⁵ As study 1 shows, this effect can be obtained in the absence of active visualisation or a learning intention, the only requirement being that the target and cue are perceived as interacting objects. Imagery mnemonics are inefficient in patients whose memory impairment has amnesic qualities, as in DAT. We therefore expected that patients with DAT would fail on the VAT. This is confirmed by studies 2 and 3. Study 2 shows that low VAT scores distinguish patients with DAT from normals, even in the preclinical phase. In addition, study 3 shows that the test distinguishes DAT from other types of dementia. We found similar sensitivities for DAT in both studies—that is, around 80% for VAT scores of 2 points or less. The specificity of this cut point for the distinction between early DAT and normalcy was 97.5% (study 2), whereas it was 69% for the distinction between DAT and other types of dementia (study 3). This difference in specificity is to be expected in view of the comparison groups used (normal elderly subjects and patients with non-DAT dementia respectively). We conclude from these findings that the VAT may offer a considerable amount of differential diagnostic information.

The design of the VAT was inspired by the spectacular learning effect of imagery mnemonics. Our results are in accordance with Crovitz's principle of encoding specificity, which holds that recall can be raised to the level of recognition if a cue is encoded along with the target at the time of presentation.³⁷ We suppose that the experiences of daily life are encoded and reproduced in similar ways, and that the integrity of these processes is reflected by the incidental learning of visual associations as measured with the VAT. If the VAT indeed reflects the integrity of everyday episodic memory, it will also be a good predictor of practical impairment.

The favourable characteristics of the VAT are probably also due to the fact that paired associate learning taxes hippocampal functions.³⁸ It is well known that the hippocampus is one of the first brain structures to deteriorate in DAT.³⁹ This explains the sensitivity of the VAT for early DAT. On the other hand, people whose memory functions are intact or relatively mildly deteriorated perform at or near the ceiling of the VAT. The memory problems in non-DAT types of dementia are often not caused by dramatic hippocampal failure but by neuropathology elsewhere in the brain. This probably leaves automatic memory processes relatively undisturbed in non-DAT dementia, but causes disorders in the effortful processes of learning and retrieval. The VAT seems to circumvent these effortful processes, which may explain its specificity.

The validity of the VAT compares favourably to recently published brief screening tests with respect to the distinction between DAT and normalcy,^40,41 as well as with respect to the distinction between DAT and other types of dementia.⁴² However, the briefest of these tests was not investigated in a non-DAT sample, whereas the one that was investigated in non-DAT patients takes more time to administer than the VAT. At first sight, these tests appear to differ from the VAT by showing that delayed recall of patients with DAT is impaired relative to immediate recall. However, it seems probable that immediate recall in these tests is derived from working memory. By erasing the material from working memory, the delay (or the distraction provided) shows that it is not encoded in long term memory. In the VAT, recall is delayed to the extent that the presentation of each item is followed by the presentation or recall of the other five items. Our findings do not offer conclusive proof for an encoding deficit in DAT, but our clinical experience with the VAT supports the possibility. The errors of patients with DAT usually indicate a total unawareness of the targets and often consist of obvious associates of the cue, such as ape-banana. On the other hand, if a patient with DAT succeeds in producing any correct responses, these are likely to be replicated even when recall is tested 30 minutes later. This points to an encoding deficit rather than accelerated forgetting.

In addition to its predictive validity, the VAT has several attractive properties as a screening test. It is not taxing, it takes very little time to administer, and it is not confounded by age, education, or depression. The only prerequisite is that the stimuli can be consciously perceived, which may not be satisfied by patients with severe attention disorders or visual impairment. Of course, we do not imply that administration of the VAT as a stand alone test is sufficient to diagnose DAT. Our study shows that the VAT may be helpful in the early diagnosis of DAT and in the differential diagnosis of dementia syndromes in general. However, more medical and neuropsychological information is needed for a reliable (differential) diagnosis. The diagnostic contribution of neuropsychology is probably greatest in cases that present with cognitive abnormalities in the absence of other neurological or somatic symptoms. In such cases one needs a more refined neuropsychological profile than offered by a short screening test. If, for example, in addition to memory tests such as the VAT we also consider tests of psychomotor speed, executive functioning, and semantic memory, then DAT and other types of dementia may be distinguished with greater precision.^43,44

In conclusion, the VAT is a promising tool for the assessment of amnesia. Besides the detection of DAT it may be useful for various other purposes, such as diagnosing amnesic syndromes with various causes, monitoring recovery from post-traumatic amnesia, and screening patients eligible for imagery mnemonics training. As the VAT was designed to identify a form of memory impairment that is not readily compensated for, it may also be a good indicator of practical impairment.